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Period for Reply 

A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- tf the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- if NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)IE This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) M Claim(s) 7-25 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-25 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

£})□ The specification is objected to by the Examiner. 

10)S The drawing(s) filed on 17 November 2003 is/are: a)|E accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
1 !)□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)0 All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. D Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 
Information Disclosure Statement 

1 . The information disclosure statement filed on 10/05/2004 fails to comply with 37 CFR 

1 .98(a)(2), which requires a legible copy of each cited foreign patent document; each non-patent 
literature publication or that portion which caused it to be listed; and all other information or that 
portion which caused it to be listed. It has been placed in the application file, but the information 
referred to therein has not been considered. The AN reference is not considered herein because 
the reference copy is incomplete. However, the examiner finds the AN reference to be relevant to 
the rejection has incorporated the entire reference in this office action. The copy of the whole 
reference can be downloaded at the URL link of 
http: //ww w .ri. emu . edu/pub_files^ 

Claim Rejections - 35 USC §102 
The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed publication in this or a 
foreign country, before the invention thereof by the applicant for a patent 

2. Claims 1-25 are rejected under 35 U.S.C. 102(a) as being anticipated by Frank Dellaert 
("Monte Carlo EM for Data-Association and its Applications in Computer Vision", September 
21, 2001, CMU-CS-01-153, School of Computer Science, Carnegie Mellon University). 

a. Referring to Claim 1, Dellaert discloses receiving image data comprising a 
plurality of photographic images of a three-dimensional scene (see figure 1.2 on page 12, 
a plurality of image of a three-dimensional building are captured); receiving user input 
indicating a plurality of corresponding features each appearing in at least two of the 
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plurality of photographic images (user indicates a plurality of corresponding features, the 
circles shown in figure 1.2); determining an error function for reverse-rendering function, 
the reverse-rendering function defining a relationship between three-dimensional 
coordinates in the three-dimensional scene and corresponding two-dimensional 
coordinates of the plurality of corresponding features (page 29, the maximum likelihood 
estimate function 2.1 is the claimed "reverse-rendering function", which defines 
relationships between 3D points and the points in the 2D images, also see equation 2.2; 
equation 2.3 is the claimed "error function"); and minimizing the error function to 
determine a solution corresponding to a global minimum of the error function, 
comprising calculating at least first derivatives of the error function using automatic 
differentiation, thereby computing intermediate solution estimates for successive 
iterations of the error function, until the solution estimates converge to the solution (see 
Appendix A, page 203, equation A.l for error minimization; and the minimization 

process is iterative as described in A. 1 continued on page 204; and the Jacobian of h() is 
H n which is the claimed "first derivatives". For convergence please see paragraph 1 in 
section A. 2 on page 204. Note the calculation of the Jacobians, the first derivatives, is 
done using automatic differentiation as described on page 211). 

b. Referring to Claim 2, Dellaert discloses wherein the determining step further 
comprises determining the error function comprising reverse-rendering parameters 
selected from group consisting of camera position, camera orientation, focal length, 
aperture size, lens distortion, and distortion of focal plane (see section A. 3 for camera 
parameters). 
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c. Referring to Claim 3, Dellaert discloses wherein the determining step further 
comprises determining the error function comprising reverse-rendering parameters 
including at least one camera position located within the three-dimensional scene (see 
section A.3). 

d. Referring to Claim 4, Dellaert discloses further comprising receiving an initial 
scene graph comprising at least a portion of an initial solution estimate (page 204, an 
initial guess 0° is the claimed "initial solution estimate"). 

e. Referring to Claim 5, Dellaert discloses wherein the receiving an initial scene 
graph step further comprises receiving the initial scene graph comprising at least one 
transform defining a relationship between a parent object and a child object (page 63, in 
the first step of algorithm, the motion estimate is included in the initial guess. Such 
motion estimate can be done using JPDAF, page 44, section 3.2.2, which comprises a 
transform to define a relationship between at least two objects). 

f. Referring to Claim 6, Dellaert discloses wherein the minimizing step further 
comprises calculating an exact Hessian of the error function (page 205, equation A.4). 

g. Referring to Claim 7, Dellaert discloses further comprising initializing at least 
selected three-dimensional coordinates of the plurality of corresponding features and 
camera parameters for the plurality of photographic images as an initial solution estimate 
(section A.3 on pages 206 and 207). 

h. Referring to Claim 8, Dellaert discloses defining a resulting scene graph for scene 
consistent with the solution (Dellaert teaches using non-linear minimization, which 
iteratively optimize the maximum-likelihood of the matching, which at the end defines a 
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resulting scene graph for scene consistent with the solution, also see section A. 3 for 
change with respect to an arbitrary parameter and change in camera parameters, and 
section 2.2 about motion model parameters. All of these parameters require the error 
minimization updated recursively to achieve a resulting scene graph), 
i. Referring to Claim 9, Dellaert discloses wherein the determining step further 
comprises determining the error function further defined by a user-selected differentiable 
relationship between user-selected parameters of the reverse-rendering function (section 
A. 6 on page 211, Dellaert teaches the parameters in automatic differentiation can be 
defined by the user to achieve desired error function minimization), 
j. Referring to Claim 10, Dellaert discloses wherein the determining step further 
comprises determining the error function defined by animation parameters to solve 
match-moving relationships between frames of a motion picture sequence (motion model 
parameters are the claimed "animation parameters"). 

k. Referring to Claim 11, Dellaert discloses wherein receiving step further comprises 
receiving the plurality of photographic images representing a time sequence, wherein the 
determining step further comprise determining the error function further defined by time 
parameters for solving match-moving relationships between frames of a motion picture 
sequence, and wherein the minimizing step further comprises minimizing the error 
function simultaneously over the frames (section 2.2 on page 28, Dellaert teaches the 
time- varying structure can be accommodated using straightforward modifications. The 
time-varying herein means motion spread through a sequence of frame images. Also see 
sections 3.1.5 and 3.3). 
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1. Referring to Claim 12, Dellaert discloses wherein the receiving image data step 
comprises receiving the photographic images comprising digital images from a digital 
camera (the photographic images shown in figure 1.2 on page 12 are taken by a digital 
camera). 

m. With regard to Claim 13, see explanation in Claim 1 . 
n. With regard to Claim 14, see explanation in Claims 2 and 4. 
o. With regard to Claim 15, see explanation in Claim 5. 
p. With regard to Claim 16, see explanation in Claim 9. 
q. With regard to Claim 17, see explanation in Claim 10. 
r. With regard to Claim 18, see explanation in Claim 6. 

s. With regard to Claim 19, see explanation in Claim 1, the algorithm in Dellaert is 
implemented on a computer. 

t. With regard to Claim 20, see explanation in Claim 14. 
u. With regard to Claim 2 1 , see explanation in Claim 16. 
v. With regard to Claim 22, see explanation in Claim 17. 
w. With regard to Claim 23, see explanation in Claim 18. 
x. With regard to Claim 24, see explanation in Claim 1 1 . 
y. With regard to Claim 25, see explanation in Claim 3. 

Conclusion 

3. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Tom Y Lu whose telephone number is (703) 306-4057. The 
examiner can normally be reached on 8:30AM-5PM. 



Application/Control Number: 10/715,869 



Page 7 



Art Unit: 2621 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached on (703) 308-5246. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



Tom Y. Lu 




